[SPARK-54415][SQL] CREATE GLOBAL TEMPORARY VIEW / TEMPORARY VIEW supports IF NOT EXISTS clause #53129

pan3793 · 2025-11-19T09:56:38Z

What changes were proposed in this pull request?

Make CREATE TEMPORARY VIEW and CREATE GLOBAL TEMPORARY VIEW work with IF NOT EXISTS, like the persisted VIEW.

Why are the changes needed?

Currently, Spark SQL supports CREATE [ OR REPLACE ] [ [ GLOBAL ] TEMPORARY ] VIEW [ IF NOT EXISTS ] view_identifier ... syntax, but the implementation forbids using CREATE [ GLOBAL ] TEMPORARY VIEW with IF NOT EXISTS, this is likely an incompleted feature.

And this was also provided as an example in the docs, but actually does not work.

https://spark.apache.org/docs/4.0.1/sql-ref-syntax-ddl-create-view.html

In addition, the error condition TEMP_TABLE_OR_VIEW_ALREADY_EXISTS's message suggests "add the IF NOT EXISTS clause", which does not work either.

spark-sql (default)> CREATE GLOBAL TEMPORARY VIEW v AS SELECT 1;
spark-sql (default)> CREATE GLOBAL TEMPORARY VIEW v AS SELECT 1;
[TEMP_TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create the temporary view `v` because it already exists.
Choose a different name, drop or replace the existing view,  or add the IF NOT EXISTS clause to tolerate pre-existing views. SQLSTATE: 42P07

I also checked Snowflake docs, which indicates support for the above cases too.

https://docs.snowflake.com/en/sql-reference/sql/create-view

Does this PR introduce any user-facing change?

Yes, improves SQL syntax

How was this patch tested?

UTs are added.

Was this patch authored or co-authored using generative AI tooling?

No.

pan3793 · 2025-11-19T09:58:06Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/GlobalTempViewManager.scala

-    if (!overrideIfExists && viewDefinitions.contains(name)) {
-      throw new TempTableAlreadyExistsException(name)
+    if (ignoreIfExists && overrideIfExists) {
+      throw QueryCompilationErrors.createViewWithBothIfNotExistsAndReplaceError()


is it okay to reuse CREATE_OR_REPLACE_WITH_IF_NOT_EXISTS_IS_NOT_ALLOWED, or better suggestion?

pan3793 · 2025-11-19T09:59:08Z

sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala

  with CTEInChildren
-  with CreateTempView {
+  with CreateTempView
+  with SessionStateHelper {


as I touch this file, I fixed the IDE index issue of the whole file by leveraging SessionStateHelper

pan3793 · 2025-11-19T09:59:29Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ddl.scala

    global: Boolean,
    provider: String,
-    options: Map[String, String]) extends LeafRunnableCommand {
+    options: Map[String, String]) extends LeafRunnableCommand with SessionStateHelper {


same here, fixes IDE-index issue

pan3793 · 2025-11-19T10:02:41Z

sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala

        finalSchemaBinding)
    } else {
-      // Disallows 'CREATE TEMPORARY VIEW IF NOT EXISTS' to be consistent with
-      // 'CREATE TEMPORARY TABLE'


the logic came from SPARK-6339 (Spark 2.0.0), but seems without a clear reason

pan3793 · 2025-11-19T10:04:07Z

sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala

+          // make sure our view doesn't change.
+          val viewIdentifier = viewType match {
+            case "GLOBAL TEMPORARY VIEW" =>
+              TableIdentifier("testView", Some(conf.getConf(GLOBAL_TEMP_DATABASE)))


global temp view must be accessed via a qualified name.

pan3793 · 2025-11-19T10:05:13Z

common/utils/src/main/resources/error/error-conditions.json

  },
+  "CREATE_OR_REPLACE_WITH_IF_NOT_EXISTS_IS_NOT_ALLOWED" : {
+    "message" : [
+      "CREATE OR REPLACE <resourceType> with IF NOT EXISTS is not allowed."


We can reuse this for other resource types, e.g. TABLE, FUNCTION, PROCEDURE

core/src/test/scala/org/apache/spark/SparkThrowableSuite.scala

pan3793 · 2025-11-19T10:13:35Z

cc @cloud-fan @LuciferYang @dongjoon-hyun

BTW, this patch targets 4.2, maybe I should make a small change in the docs to remove the illegal example SQL first, to fix the current docs for all existing active branches?

amaliujia · 2025-11-19T21:27:24Z

Curious what is the current behaivor of CREATE GLOBAL TEMPORARY VIEW / TEMPORARY VIEW if the view already exists? Does the current behavior does a replacement or silently skip execution or even throw errors to say the view already exist?

Whatever we are extending here, we need to consider the backward compability with exsiting behavior (or it becomes a behavior change which we need a way to take care of).

pan3793 · 2025-11-20T03:12:29Z

@amaliujia no breaking behavior here, use 4.1.0-preview4

spark-sql (default)> CREATE GLOBAL TEMPORARY VIEW v AS SELECT 1;
spark-sql (default)> CREATE GLOBAL TEMPORARY VIEW v AS SELECT 1;
[TEMP_TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create the temporary view `v` because it already exists.
Choose a different name, drop or replace the existing view,  or add the IF NOT EXISTS clause to tolerate pre-existing views. SQLSTATE: 42P07

and as you can see, the error message even suggests "add the IF NOT EXISTS clause"

amaliujia · 2025-11-20T03:50:12Z

@amaliujia no breaking behavior here, use 4.1.0-preview4

spark-sql (default)> CREATE GLOBAL TEMPORARY VIEW v AS SELECT 1;
spark-sql (default)> CREATE GLOBAL TEMPORARY VIEW v AS SELECT 1;
[TEMP_TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create the temporary view `v` because it already exists.
Choose a different name, drop or replace the existing view,  or add the IF NOT EXISTS clause to tolerate pre-existing views. SQLSTATE: 42P07

and as you can see, the error message even suggests "add the IF NOT EXISTS clause"

Then this PR is just extending the feature which makes perfectly sense.

dongjoon-hyun · 2025-11-20T15:35:47Z

Thank you for pinging me, @pan3793 . +1 for targeting 4.2.0. For docs, maybe for 4.1.0.

BTW, this patch targets 4.2, maybe I should make a small change in the docs to remove the illegal example SQL first, to fix the current docs for all existing active branches?

holdenk · 2025-11-20T18:50:25Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/GlobalTempViewManager.scala

      overrideIfExists: Boolean): Unit = synchronized {
-    if (!overrideIfExists && viewDefinitions.contains(name)) {
-      throw new TempTableAlreadyExistsException(name)
+    if (ignoreIfExists && overrideIfExists) {


So why both ignoreIfExists and overrideIfExists and not just ignoreIfExists?

@holdenk

overrideIfExists SQL equivalent: OR REPLACE

ignoreIfExists SQL equivalent: IF NOT EXISTS

CREATE [ OR REPLACE ] GLOBAL TEMP VIEW [ IF NOT EXISTS ] ...

obviously, OR REPLACE and IF NOT EXISTS can not co-exist, except for this, all other combinations are valid.

Snowflake has the same behavior

https://docs.snowflake.com/en/sql-reference/sql/create-view#general-usage-notes

The OR REPLACE and IF NOT EXISTS clauses are mutually exclusive. They can’t both be used in the same statement.

…orts IF NOT EXISTS clause

github-actions bot added SQL STRUCTURED STREAMING CORE labels Nov 19, 2025

pan3793 commented Nov 19, 2025

View reviewed changes

core/src/test/scala/org/apache/spark/SparkThrowableSuite.scala Outdated Show resolved Hide resolved

holdenk reviewed Nov 20, 2025

View reviewed changes

pan3793 added 2 commits November 24, 2025 11:48

[SPARK-54415][SQL] CREATE GLOBAL TEMPORARY VIEW / TEMPORARY VIEW supp…

44ac58c

…orts IF NOT EXISTS clause

restore docs

5e6ad0e

pan3793 force-pushed the SPARK-54415 branch from 7fd37c1 to 5e6ad0e Compare November 24, 2025 03:48

github-actions bot added DOCS and removed CORE labels Nov 24, 2025

[SPARK-54415][SQL] CREATE GLOBAL TEMPORARY VIEW / TEMPORARY VIEW supports IF NOT EXISTS clause #53129

Are you sure you want to change the base?

[SPARK-54415][SQL] CREATE GLOBAL TEMPORARY VIEW / TEMPORARY VIEW supports IF NOT EXISTS clause #53129

Uh oh!

Conversation

pan3793 commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

Uh oh!

pan3793 Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

pan3793 Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pan3793 Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

pan3793 Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pan3793 Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

pan3793 Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pan3793 commented Nov 19, 2025

Uh oh!

amaliujia commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pan3793 commented Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amaliujia commented Nov 20, 2025

Uh oh!

dongjoon-hyun commented Nov 20, 2025

Uh oh!

holdenk Nov 20, 2025

Choose a reason for hiding this comment

Uh oh!

pan3793 Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pan3793 commented Nov 19, 2025 •

edited

Loading

pan3793 Nov 19, 2025 •

edited

Loading

pan3793 Nov 19, 2025 •

edited

Loading

amaliujia commented Nov 19, 2025 •

edited

Loading

pan3793 commented Nov 20, 2025 •

edited

Loading

pan3793 Nov 21, 2025 •

edited

Loading